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ABSTRACT 

In  this  paper,  an  automated  change  detection  technique  is  presented  that  compares  new  and  historical  seafloor 
images  created  with  sidescan  synthetic  aperture  sonar  (SAS)  for  changes  occurring  over  time.  The  method 
consists  of  a  four  stage  process:  a  coarse  navigational  alignment;  fine-scale  co-registration  using  the  scale  invariant 
feature  transform  (SIFT)  algorithm  to  match  features  between  overlapping  images;  sub-pixel  co-registration  to 
improves  phase  coherence;  and  finally,  change  detection  utilizing  canonical  correlation  analysis  (CCA).  The 
method  was  tested  using  data  collected  with  a  high-frequency  SAS  in  a  sandy  shallow- water  environment.  By 
using  precise  co-registration  tools  and  change  detection  algorithms,  it  is  shown  that  the  coherent  nature  of  the 
SAS  data  can  be  exploited  and  utilized  in  this  environment  over  time  scales  ranging  from  hours  through  several 
days. 

Keywords:  automated  change  detection,  canonical  correlation  analysis,  coherent  change  detection,  co-registration, 
synthetic  aperture  sonar,  SAS 


1.  INTRODUCTION 

Detecting  areas  of  change  in  multiple  images  of  the  same  scene  captured  at  different  times  is  of  interest  to  vastly 
different  disciplines.  Applications  of  change  detection  include:  remotely  sensed  satellite  imageryjIGI  synthetic 
aperture  radar  (SAR)P  medical  imaging®^  industry  applications J3®  and  driver  assistance  systems HEO  For 
sonar  and  radar  systems,  change  detection  is  the  process  by  which  regions  of  interest  are  identified  through  the 
comparison  of  current  data  with  historical. 

In  image-based  change  detection,  entire  image  portions  are  compared  to  identify  regions  of  change.  Myers,  et 
al.,nI  used  several  change  detection  methods  to  calculate  overall  degree  of  similarity  between  two  sidescan  sonar 
images.  Typically,  change  detection  is  applied  to  situations  where  the  same  area  is  to  be  repeatedly  monitored, 
such  as  surveys  for  port  and  harbor  security.  This  change  detection  technique  would  increase  detection  capability 
and  reduce  the  workload  of  a  human  operator  tasked  with  identifying  anomalies  in  sidescan  sonar  images. 

Techniques  for  image-correlation  automated  change  detection  (ACD)  have  been  under  development  by  the 
SAR  community  since  at  least  the  1990sJ21and  procedures  to  fuse  scene  changes  derived  from  segmented  features 
with  pixel  or  parcel  based  change  maps  have  recently  been  explored.12^  In  the  sonar  research  community,  contact 
correlation  ACD  approaches  were  developed  first.13  This  was  due  to  a  number  of  factors,  such  as  challenges  in 
the  trajectory  control,  accurate  georegistration,  the  complexity  of  the  propagation  medium,  and  the  radiometric 
inconsistencies  of  conventional  sidescan  sonars.  Only  in  recent  years,  with  the  advent  of  synthetic  aperture  sonars 
(SAS)  and  actively  navigated  platforms,  have  image  correlation  methods  been  shown  to  be  feasible.^1- 

The  inspiration  for  the  current  work  comes  from  an  extension  of  the  coherent-based  change  detection  results 
using  canonical  correlation  analysis  (CCA)  described  by  Azimi-Sadjadi  and  Srinivasan^  G-Michael  and  Tuckei®0 
and  Sternlicht  and  G-Michaelf^  where  the  preliminary  studies  were  performed  on  simulated  SAR  and  SAS 
imagery.  The  motivation  behind  CCA  comes  from  recent  success  using  it  as  a  target  detection  scheme  in  SAS 
imagery.^  In  this  paper,  we  develop  a  complete  ACD  algorithm  that  combines  the  use  of  the  Scale  Invariant 
Feature  Transform  (SIFT)  algorithm21  EU  as  a  co-registration  tool  with  the  change  detection  method  of  using 
CCA.  In  CCA,  the  correlation  is  formed  between  two  sonar  images  from  repeat  passes  over  the  ocean  floor.  From 
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Figure  1:  Block  diagram  for  Image-based  Automated  Change  Detection  (ACD)  processing. 

this  correlation,  coherence  (or  incoherence)  can  be  measured  and  used  to  determine  what  has  changed  between 
the  two  sonar  images. 

Our  methods  are  tested  on  SAS  imagery  collected  with  the  high-frequency  projector  of  the  Small  Synthetic 
Aperture  Mine-hunter  (SSAM)  system  developed  by  the  Naval  Surface  Warfare  Center  Panama  City  Division 
and  the  Applied  Research  Laboratory,  Pennsylvania  State  University.^  The  SSAM  system  includes  an  advanced 
sidescan  sonar  that  is  mounted  on  a  Woods  Hole  Oceanographic  Institution  REMUS  600  autonomous  undersea 
vehicle  (AUV).  It  is  able  to  achieve  fine-detail  high-resolution  SAS  imaging  of  the  seabed  using  a  high-frequency 
(>100  kHz)  projector. 

This  paper  is  organized  as  follows:  Section  [2]  presents  an  overview  of  the  proposed  image-based  ACD  process. 
Section  [3]  provides  detailed  explanation  of  our  co-registration  method.  Section  [4]  describes  the  CCA  algorithm 
and  how  they  measure  change.  Section  [5]  presents  performance  results  of  our  co-registration  and  change  detection 
methods.  Finally,  Section  [6]  provides  conclusions,  discussion,  and  directions  for  future  research. 

2.  AN  OVERVIEW  OF  ACD  PROCESS 

Image-based  change  detection  is  carried  out  through  the  comparison  of  reference  and  repeat-pass  images  that 
correspond  to  the  same  geographical  location,  but  are  collected  at  different  times.  The  automation  of  this 
task  involves  three  major  components:  historical  data  retrieval,  reference  and  repeat-pass  co-registration,  and 
detection  of  changes.  A  block  diagram  of  ACD  processing  is  illustrated  in  Fig  |TJ 

A  historical  database  comprised  of  SAS  images  with  their  associated  geographical  locations  provides  the 
baseline  or  reference.  Given  a  specific  repeat-pass  image,  images  from  this  database  that  roughly  correspond  to 
the  same  geographical  location  are  retrieved  for  use  in  the  following  co-registration  and  change  detection  steps. 
All  image  geographical  information  for  both  the  database  and  repeat-pass  images  are  collected  from  the  REMUS 
600  on-board  navigation  system  and  is  accurate  up  to  the  fidelity  of  this  system,  which  is  comprised  of  a  Doppler 
velocity  log  and  GPS-aided  inertial  navigation  system. 

3.  CO-REGISTRATION 

Image  co-registration  is  a  key  step  in  the  ACD  process.  It  provides  the  image  registration  necessary  for  image 
comparison  and  change  detection.  For  our  co-registration  method,  we  propose  a  three  stage  process:  navigation 
alignment,  fine-scale  co-registration,  and  sub-pixel  co-registration. 

3.1  Navigation  Alignment 

After  the  identification  and  loading  of  the  historical  data  corresponding  to  the  repeat-pass  image,  a  purely 
navigational  coarse  alignment  is  our  first  step  toward  co-registration  of  the  images.  The  process  involves  using 
the  nominal  paths  for  the  reference  and  repeat-pass  images.  The  nominal  path  for  an  image  is  the  straight  line 
path  taken  from  the  image  starting  location  to  the  end.  For  the  repeat-pass  image  we  construct  a  linear  mapping 
from  latitude  and  longitude  sample  location  to  their  relative  pixel  location  by  solving  the  following  system  of 
equations: 
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where  the  size  of  the  repeat-pass  data  is  m  x  n,  {xi,  x%,  *3}  are  the  latitude  and  longitude  of  the  endpoints  of 
the  repeat-pass  data,  and  {ci,  C2}  are  the  linear  mapping  coefficients  that  we  are  solving  for.  Using  this  mapping 
we  can  calculate  the  reference  data  pixel  values  relative  to  the  repeat-pass  data,  which  allows  interpolation  of 
the  repeat-pass  data  to  the  reference  data  relative  to  pixel  location.  This  projects  the  repeat-pass  data  on  to  the 
reference,  thereby  removing  most  navigation  differences  between  reference  and  repeat-pass  images. 


3.2  Fine-Scale  Co-Registration 

For  fine-scale  co-registration  we  apply  the  scale  invariant  feature  transform  (SIFT)  algorithnEDE9  to  the  am¬ 
plitude  image  data.  The  SIFT  algorithm  detects  and  extracts  distinct  features  from  images  which  can  be  used 
to  perform  matching  between  overlapping  images.  The  SIFT  features  computed  for  each  keypoint  are  gradient 
magnitudes  and  orientations  that  are  sampled  around  the  keypoint  location.  In  order  to  achieve  rotation  in¬ 
variance,  gradient  orientations  are  rotated  relative  to  the  keypoint  orientation.  SIFT  keypoints  and  features  are 
computed  for  both  navigation-aligned  reference  and  repeat-pass  data.  Keypoint  matching  between  reference  and 
repeat-pass  images  is  done  by  computing  minimum  Euclidean  distance  between  SIFT  features.  In  order  to  reject 
weak  keypoints,  matches  where  the  second  closest  neighbor  is  within  1.5  times  the  closest  match  are  rejected. 

Differencing  the  matched  keypoints  typically  results  in  a  collection  of  noisy  estimates  for  translational  shifts. 
In  order  to  approximate  the  true  shift,  we  first  remove  outliers  by  iteratively  removing  shift  estimates  that  are 
outside  three  standard  deviations  from  the  mean. 

Once  outliers  have  been  removed,  our  final  shift  estimate  is  computed  as  a  weighted  average  of  the  remaining 
shift  estimates.  Specifically,  letting  {«i,  S2, . . . ,  sjv}  be  our  remaining  collection  of  shift  estimates  and  cv/  be  the 
Euclidean  distance  from  s,  to  Sj.  We  compute  the  weights,  {wi,W2,  ■  ■  ■  ,wj\ r} ,  as 
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For  weights  where  riiiiij  Cj?  is  zero,  we  set  the  weight  value  to  twice  the  maximum  of  the  defined  weights.  The 
weights  are  then  normalized, 
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and  the  image  shift  is  estimated  as 
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An  alternative  method  for  estimating  the  image  shift  is  to  fit  a  Gaussian  distribution  to  the  remaining  shift 
estimates.  The  shift  is  then  approximated  as  the  mean  of  the  fitted  Gaussian  distribution.  This  method  works 
well  when  there  is  a  large  number  of  shift  estimates  to  use  for  fitting.  However,  we  have  found  that  with  more 
difficult  environments  and  longer  elapsed  times  between  passes,  the  quality  of  the  resulting  estimate  is  not  as 
good  as  the  estimate  obtained  with  the  weighted  average. 

With  the  fine  co-registration  shift  offsets  determined,  both  reference  and  repeat-pass  images  are  cropped, 
such  that  only  common  pixel  locations  for  the  two  images  are  passed  to  the  change  detection  algorithm. 


3.3  Sub-Pixel  Co-Registration 

Following  fine-scale  co-registration  the  average  scene  displacement  is  ideally  on  the  order  of  a  single  pixel.  How¬ 
ever,  local  registration  error  can  be  significantly  higher  than  the  average,  particularly  if  the  deviation  between 
platform  trajectories  is  significant  or  if  rotation  is  present.  This  spatially-variant  mismatching  can  have  a  very 
large  (and  negative)  effect  on  coherence  estimates  between  scenes.  The  problem  is  similar  to  that  encountered 
when  co-registering  wide-baseline  interferometric  channels.24  25  Our  sub-pixel  co-registration  works  as  an  adap¬ 
tive  correlation  technique,  where  the  offset  between  images  is  a  function  of  the  X  —  Y  location  within  the  image. 
It  uses  a  quality  metric  to  estimate  whether  coherent  or  intensity  based  correlation  will  result  in  optimal  scene 
shift  measurements.  It  works  in  such  a  way  that,  if  the  average  correlation  values  computed  over  the  overlapping 
portions  of  the  projection  of  the  sonar  beam  on  the  sea  floor  dip  below  a  threshold,  then  intensity-based  correla¬ 
tion  values  are  calculated  and  a  smoothness  metric  is  evaluated  for  the  best  shift  estimate  results.  The  method 
which  results  in  the  best  shift  estimate  is  selected.  A  neighborhood  of  pixels  with  a  fixed  block  size  is  chosen 
to  allow  for  large  offsets  between  images,  while  still  maintaining  a  reasonably  high  correlation  and  the  ability 
to  generate  a  spatially  variant  solution.  The  resolution  of  the  computed  interferogram  (a  phase  difference  map 
implemented  using  the  product  of  the  reference  image  matrix  and  the  complex  conjugate  of  the  co-registered 
repeat-pass  matrix)  and  the  resulting  inter-scene  coherence  map  can  be  improved  by  sub-pixel  co-registration, 
because  all  the  points  in  the  corresponding  neighborhoods  overlap.  Once  the  images  have  been  aligned,  there 
is  a  simple,  one-to-one  linear  mapping  of  pixels  between  scenes,  which  is  used  to  flatten  the  interferogram  and 
improve  the  correlation  between  scenes. 

Specifically,  sub-pixel  co-registration  involves  two  steps:  warping  the  complex  values  in  the  scene  to  align 
pixels  on  a  local  level,  and  flattening  the  interferogram  such  that  the  phase  component  due  to  the  flat-earth 
phase  trend  is  removed.  In  the  current  examples,  sub-pixel  co-registration  is  accomplished  via  the  steps  shown 
in  Algorithm  [I] 


Algorithm  1  Sub-pixel  co-registration 

Step  1.  Candidate  scenes  are  autofocused 

Step  2.  Large-neighborhood  (e.g.  25  x  25  pixel),  non-overlapping  patch  correlation  and  parabolic  filtering  is  used  to 
estimate  local  patch  alignment  errors  to  pixel  level 

Step  3.  Using  an  iterative,  weighted  least-squares  approach,  the  X  and  Y  shifts  are  independently  fit  to  an  appropriate 
dictionary  of  functions 

Step  4.  A  two-dimensional  warping  function  is  defined  using  the  coefficients  calculated  in  step  [3]  and  applied  to  the 
matching  image 

Step  5.  Small-neighborhood  (10  x  10  pixel)  patch-correlation  is  used  to  generate  an  interferogram 

Step  6.  A  weighted  minimum-discontinuity  unwrapping  algorithm  is  applied  to  the  interferogram 

Step  7.  A  dictionary  of  functions  defined  by  the  estimated  phase  resulting  from  navigational  deviations  between  repeat 
passes  is  fit  using  an  iterative  least-squares  approach  to  the  unwrapped  phase 

Step  8.  An  interferometric  flattening  function  is  generated  using  the  calculated  dictionary  coefficients 

Step  9.  The  flattening  function  is  used  to  calculate  a  phase  which  is  subtracted  from  the  matching  image 


The  full  details  of  some  of  these  steps  are  fairly  involved  and  lie  beyond  the  scope  of  this  paper;  however,  it 
is  similar  to  the  algorithm  described  in  Saebo,  et  alP^l  Particular  steps  merit  additional  attention.  In  some  situ¬ 
ations  one  can  observe  navigation-induced  focusing  defects,  which  can  significantly  degrade  correlation  between 
neighborhoods,  even  if  alignment  was  fairly  good.  A  non-parametric  method  similar  to  that  used  in  multi-squint 
processing^6  but  highly  local  and  modified  to  work  with  wide-beam/wide-band  systems,  was  used  to  sharpen  the 
focus  of  the  beamformed  scenes  in  step[l]  The  correlation  coefficients  calculated  in  steps  [2]  and  [5]  can  be  used  for 
inter-scene  coherence  estimates.  A  qualitative  comparison  of  the  correlation  coefficients  after  warping  and  phase 
flattening  indicates  that  the  primary  enhancement  in  coherence  occurs  after  the  warping  step.  Typically,  unless 
the  baseline  deviation  is  significant  enough  to  produce  strong  local  curvature  in  the  phase  flattening  function, 
the  phase  correction  is  approximately  planar  and  has  minimal  effect  on  the  calculated  coherence.  Step  [7]  results 
in  a  heave  and  sway  solution  relating  the  difference  in  vehicle  trajectory  between  passes,  and  in  conjunction  with 
the  interferogram  can  be  used  to  make  repeat-pass  bathymetry  estimates. 


4.  CHANGE  DETECTION 

For  change  detection  we  utilize  Canonical  Correlation  Analysis  (CCA)P  CCA  is  a  multivariate  statistical 
method27  that  determines  the  linear  dependence  (or  coherence)  between  two  data  channels,  which  in  this  case 
correspond  to  reference  and  repeat-pass  SAS  images.  The  CCA  method  not  only  determines  the  amount  of 
dependence  (or  independence)  between  two  data  channels  but  also  extracts  a  subset  of  most  coherent  change 
features  from  the  two  channels.  This  is  accomplished  by  linearly  mapping  the  two  data  channels  to  their  canonical 
coordinates,  the  first  few  of  which  capture  maximal  inter-channel  coherence. 

Let  us  assume  that  the  two-channel  complex  data  vectors  are  represented  by  x  £  C”  and  y  £  Cm,  then  the 
canonical  correlations  and  coordinates  are  obtained  by  finding  the  singular  values  and  vectors  of  the  coherence 
matrix P7  which  is  defined  as 


C  =  E[{R-^x){R~^y)H], 


where  (• )H  is  the  Hermitian  operation,  E[-]  is  the  expectation  operation,  and  Rxx  and  Ryy  are  the  covariance 
matrices  of  x  and  y  channels,  respectively.  The  canonical  coordinates  of  x  and  y  are  defined  as 


u  =  FHR  ^2x 

where  F  and  G  are  the  mapping  matrices  containing  the  left  and  right  singular  vectors  of  the  coherence 
matrix  C,  respectively.  The  canonical  coordinate  vectors  m  and  v  share  the  diagonal  cross-covariance  matrix 

K  =  RUV  =  FHCG  (4) 

known  as  the  canonical  correlation  matrix  of  canonical  correlations  ki,i  £  [1,  m],  which  are  the  singular  values 
of  matrix  C. 

Now,  the  linear  dependence  between  x  and  y  channels  can  be  measured  by 

N 

L  =  [](!  -  *?)  0  <  L  <  1. 

i= 1 

This  measure  takes  the  value  0  if  there  is  perfect  linear  dependence  between  x  and  y;  while  L  =  1  corresponds 
to  the  case  when  x  and  y  are  linearly  independent.  The  coherence  measure  between  x  and  y  is  given  by 

H  =  1  —  L  0  <  H  <1.  (5) 

That  is,  the  channels  x  and  y  are  perfectly  coherent  if  H  =  1,  and  non-coherent  if  H  =  0. 

Though  the  dominant  canonical  coordinates  capture  maximal  coherence,  the  subdominant  ones  maximize  the 
divergence  between  the  two  channels  x  and  yUmam  In  other  words,  the  subdominant  canonical  coordinates 
capture  most  of  the  coherent  change  features  between  the  two  channels  with  a  minimum  dimensional  feature  set. 
The  coherent  change  information  between  canonical  coordinates  v  and  u  can  be  calculated  using  the  residual, 
v  —  Km,  owing  to  the  fact  that  Km  brings  the  most  coherent  information  that  the  second  channel  y  carries  about 
the  first  channel  x.  Now  that  the  subdominant  canonical  coordinates  capture  the  coherent  change  information 
(under  the  constraint  of  maximum  rate)  retaining  the  last  few  components  of  the  difference  vector  v  —  Km,  is 
sufficient.  This  can  be  performed  mathematically  by  using  Ip(v  —  Km),  where  Ip  is  the  m  x  m  diagonal  matrix 
with  the  first  m  —  p  diagonal  entries  being  zero  and  the  rest  being  unity.  Moreover,  we  can  convert  the  difference 
vector  back  to  the  image  space  from  the  canonical  space  using, 

d,  =  R}JyGIp{v  —  Km).  (6) 

For  a  more  complete  description  of  the  CCA  change  detection  method  for  SAS  imagery  the  reader  is  referred 
toP 
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Table  1:  Percentage  of  images  with  estimated  shift  greater  than  true  shift  values  for  a  feature  rich  environment 
(set  1)  and  a  feature  poor  environment  (set  2). 

5.  DATA  AND  RESULTS 

To  demonstrate  the  feasibility  of  our  ACD  approach  we  operated  on  SAS  data  collected  with  the  high-frequency 
aperture  of  the  SSAM  system  for  a  variety  of  measurement  scenarios.  We  start  with  a  qualitative  demonstration 
of  our  approaches  to  co-registration  and  change  detection,  followed  by  a  quantification  of  this  performance  by 
experiments  that  measure  performance  of  both  co-registration  and  change  detection. 

In  Fig.  [2] we  show  a  qualitative,  example  of  the  result  from  the  automated  co-registration  technique  described 
in  Section  [3]  applied  in  a  flat  sandy  region.  Typically  co-registration  in  such  regions  is  more  difficult  due  to 
temporal  changes  of  the  base  image  structure.  In  this  case,  with  a  short  time  latency  of  2  hours  between  passes, 
successful  co-registration  was  possible.  We  were  able  to  achieve  high  precision  automated  co-registration  in  this 
sandy  environment  for  time  latencies  of  up  to  72  hours,  then  with  more  difficulty  for  longer  time  periods. 

The  result  of  applying  the  CCA  change  detection  algorithm  to  the  co-registered  images  shown  in  Fig.  [2]  can 
be  seen  in  Fig.  [3]  In  this  figure  the  new  object  is  clearly  identified  by  the  low-level  region  of  the  coherence  map. 
Overall,  we  see  a  high  level  of  coherence  through  out  the  center  of  this  example  and  a  low  level  at  near  and  far 
range. 

5.1  Robustness  of  Co- Registration  Algorithm 

In  order  to  quantify  the  robustness  of  our  co-registration  algorithm,  we  constructed  an  evaluation  set  from  the 
reference  pass  for  data  sets  1  and  2  by  adding  noise  and  shifting  by  a  known  quantity.  The  exact  error  of  the 
co-registration  algorithm  can  then  be  calculated  as  the  Euclidean  distance  between  the  known  image  shift  and 
the  shift  estimated  by  co-registration.  The  noise  added  was  zero  mean  complex  Gaussian.  Three  levels  of  noise 
were  considered  with  variance  levels  of  1.0,  1.5,  and  2.0  times  the  variance  of  the  data.  Each  image  was  shifted 
down  and  right  by  20,  50,  100,  150,  and  200  pixels.  Table  [T]  lists  the  percentage  of  images  with  estimated  shift 
greater  than  the  true  shift  values.  Almost  half  of  all  the  errors  are  less  than  20  pixels  in  size,  while  most  of 
the  other  errors  are  greater  then  200  pixels  in  size.  In  general,  the  size  of  the  shift  difference  does  not  impact 
co-registration  performance  for  the  shifts  considered.  For  low  and  moderate  noise,  co-registration  works  well 
with  typically  less  than  a  pixel  of  distance  error  in  co-registration.  For  high  noise,  co-registration  performs  well 
for  some  images.  When  it  does  fail  the  estimated  shift  is  vastly  different  from  the  true  shift.  These  extremely 
large  co-registration  failures  can  likely  be  detected  and  flagged  for  operator  intervention.  The  percentages  in 
Table  [I]  also  indicates  the  relative  difficulty  posed  by  the  two  environments.  For  the  all-sand  environment  (data 
set  2),  twice  as  many  large  co-registration  failures  occur  at  high  noise,  compared  to  the  feature  rich  environment 
(data  set  1). 

5.2  Robustness  of  Change  Detection  Algorithm 

In  order  to  quantify  the  robustness  of  our  change  detection  algorithm,  we  performed  a  statistical  analysis  on  the 
measured  temporal  coherence  of  seafloor  backscatter.  A  detailed  comparison  of  correlation  statistics  is  carried 
out  based  on  computation  of  the  CCA  coherence  measure,  H( Eq.  [5])  over  data  sets  collected  at  site  2.  The 
duration  of  the  data  collection  was  approximately  72  hours  over  this  predominantly  flat  sand  environment.  In 
Fig-0  a  box  plot  of  the  coherence  distribution  for  the  collected  images  in  set  1  is  shown.  The  coherence  was 


(a)  Reference 


(b)  Repeat  pass 

Figure  2:  Automated  co-registration  results  collected  from  data  set  2.  A  new  object  is  present  in  the  center  of 
the  repeat-pass  image  (b). 
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Figure  3:  Coherence  maps  for  co-registered  images  shown  in  Fig.  fusing  CCA  (a).  Both  maps  range  in  value 
from  0  to  1,  with  1  or  red  indicating  high  coherence  and  0  or  blue  indicating  low  coherence. 


computed  between  reference  and  repeat-pass  data  sets  that  are  co-registered  using  the  automated  technique.  The 
abscissae  in  Fig.[4]indicate  the  time  interval  of  data  collection  for  each  repeat-pass  leg.  The  boxes  in  these  figures 
represent  the  middle  50%  of  the  coherence  distribution,  with  upper  and  lower  hinges  showing  the  75th  and  25th 
percentiles.  The  whiskers  are  extended  to  1.5  times  the  difference  between  these  percentiles  (the  inter-quartile 
range),  showing  that  the  majority  of  data  falls  between  the  ends  of  the  whiskers.  The  horizontal  red  line  through 
each  box  represents  the  median  of  the  data  within  the  box.  The  red  crosses  outside  of  the  whiskers  represent 
individual  outliers. 

As  the  correlations  vary  considerably  from  one  run  to  another  over  time,  the  box  plot  allows  us  to  better 
visualize  the  overall  trend  of  coherence  as  time  elapses.  We  observe  a  decaying  of  coherence  over  time  which 
corresponds  to  temporal  decorrelation.  The  first  five  repeat-passes  were  collected  along  1— hour  intervals.  The 
sixth  and  the  seventh  repeat-passes  were  collected  approximately  72  hours  later. 

We  determined  the  percentage  of  strong  correlations  and  defined  strong  coherence  as  a  value  greater  than 
65%.  The  percentage  of  strong  correlations  should  be  regarded  as  an  indication  of  the  ability  to  perform 
image-based  change  detection.  From  Fig.  [4]  we  see  that  the  data  is  highly  coherent  within  the  first  few  hours 
with  the  exception  of  the  data  from  the  first  hour  where  the  vehicle  altitude  changed  significantly  between  the 
reference  and  repeat-pass  data.  Removing  the  portion  of  the  image  associated  with  the  water  column  prior  to 
running  the  data  through  the  change  detection  algorithm  should  mitigate  this  problem.  Overall,  the  general 
decay  of  the  coherence  over  time  generally  agrees  with  the  observations  of  Lyons  and  BrownPS  who  quantify 
the  temporal  decorrelation  of  seafloor  roughness  and  estimate  scene  decorrelation  by  calculating  the  complex 
correlation  coefficient  for  pairs  of  SAS  images  generated  using  data  collected  with  a  rail-based  system.  The  last 
two  datasets  in  figures  [4]  represent  data  collection  approximately  72  hours  after  the  reference  pass.  They  exhibit 
a  much  lower  coherence,  indicating  that  this  sandy  environment,  lacking  stable  features,  presents  a  challenge  for 
change  detection  over  large  time  scales. 

6.  CONCLUSIONS  AND  DISCUSSION 

In  this  paper,  we  presented  and  demonstrated  a  new,  robust,  and  precise  seabed  change  detection  method 
for  synthetic  aperture  sonar  that  automatically  compares  seafloor  images  and  detects  changes  over  time.  Our 


Figure  4:  Coherence  box  plot.  Sample  box  plots  of  the  measured  coherence  of  seafloor  acoustic  backscatter,  H , 
over  time.  Seven  repeat-pass  data  sets  are  compared  to  the  reference  data  set  with  latencies  ranging  from  1  hour 
to  72  hours. 


study  primarily  focused  on  the  automation  of  the  scene  co-registration  and  change  detection  methods.  The  ACD 
method  consists  of  a  four  stage  process:  1)  Coarse  navigational  alignment  that  relates  and  approximates  the  pixel 
location  of  reference  and  repeat-pass  datasets;  2)  Fine-scale  co-registration  using  the  SIFT  algorithm  to  match 
features  between  overlapping  datasets;  3)  Sub-pixel  co-registration  that  improves  the  coherent  change  detection 
process;  4)  Coherent  change  detection  utilizing  the  CCA  and  TIS  algorithms  to  detect  changes.  The  ACD 
method  was  tested  using  data  collected  with  a  high-frequency  synthetic  aperture  sonar  in  a  sandy  shallow-water 
environment,  subject  to  currents  and  tidal  changes.  By  using  precise  co-registration  tools  and  change  detection 
methods  that  exploit  phase  as  well  as  amplitude  levels  of  the  backscattered  signals,  it  was  shown  that  coherent 
change  detection  can  be  utilized  in  this  environment  over  time  scales  ranging  from  hours  through  several  days. 

Robustness  of  the  fine-scale  co-registration  algorithm  was  demonstrated  by  testing  its  efficacy  with  image 
pairs  that  have  been  shifted  and  contaminated  with  noise.  As  applied  to  the  data  sets  collected  it  was  found  that 
the  SIFT  algorithm  was  robustly  able  to  co-register  these  contaminated  image  pairs,  with  the  process  breaking 
down  only  for  high-noise  exemplars.  This  breakdown  was  more  pronounced  for  the  feature-poor  environment. 
Analysis  of  the  distribution  of  the  CCA  coherence  measure  over  datasets  collected  with  different  time  latencies 
showed  that  coherency  between  image  pairs  that  were  co-registered  with  the  fine-scale  SIFT  algorithm  degraded 
over  time;  however,  the  sub-pixel  co-registration  algorithm  restored  some  of  the  lost  coherence  in  many  cases 
sufficiently  to  use  the  phase  portion  of  the  data  in  assessment  of  scene  change. 

Determination  of  seabed  change,  with  levels  of  precision  and  robustness  sufficient  for  alerting  to  the  introduc¬ 
tion  of  small  man-made  objects,  is  an  extremely  challenging  problem.  This  paper  demonstrated  the  potential  of 
using  automated  image-based  change  detection  techniques,  made  possible  with  the  advent  of  synthetic  aperture 
sonars  and  actively  navigated  undersea  vehicles.  The  advantage  of  utilizing  the  phase  component  of  these  co¬ 
herent  sonars  is  that  a  detected  change  in  phase  between  scenes  indicates  a  change  in  bottom  relief,  even  when 
there  is  negligible  change  in  the  amplitude  of  backscattered  energy.  This  can  be  useful  for  investigating  changes 
in  topography  or  alerting  to  the  presence  of  an  acoustically  chameleonic  object.  While  the  utility  of  exploiting 
changes  in  signal  phase  degrades  over  time,  with  time  scales  dependent  on  the  environment,  the  capability  of 
using  image-based  tools  such  as  these  to  conduct  incoherent  change  detection  for  time  scales  on  the  order  of 
weeks  has  been  demonstrated.?? 

A  number  of  emerging  tools  exist  to  detect  seabed  changes  in  multi-temporal  sidescan  sonar  images:  auto¬ 
mated  image  correlation  -  both  coherent  and  incoherent,  automated  contact  correlation,  as  well  as  interactive 


tools  to  assist  a  sonar  operator.  For  the  near  future  performing  seabed  change  detection  over  a  variety  of  en¬ 
vironments  will  likely  require  exploiting  a  number  of  these  techniques  -  utilizing  certain  techniques  for  specific 
environments  or  combining  a  number  of  tools  in  parallel.  The  next  steps  in  this  research  include:  segmentation  of 
detected  changes  to  assess  the  nature  and  characteristics  of  detected  regions  of  interest;  reduction  and  rejection 
of  clutter  regions  such  as  the  water  column  and  areas  of  decorrelation  due  to  environmental  processes;  analytical 
assessment  of  which  environments  are  most  appropriate  for  this  technique  as  well  as  effective  combination  with 
complementary  techniques;  and  finally,  the  fast  implementation  of  these  image  analysis  algorithms  on  computer 
platforms. 
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